Hospital Rankings and Health Equity.

This Viewpoint examines US News & World Report's approach to evaluating and publicly reporting hospital performance in various aspects of health equity as well as describes several novel equity measures published as part of its "Best Hospitals" rankings program.

Benchmarking in Academic Physical Therapy Using the PT-GQ Survey: Wave 2 Update With Application to Accreditation Reporting.

OBJECTIVE: The Benchmarking in Academic Physical Therapy study uses the Physical Therapist-Graduation Questionnaire (PT-GQ) survey to develop comprehensive performance benchmarks for physical therapist education. These benchmarks facilitate interprofessional comparisons and have application to accreditation self-study reporting. The purpose of this study is to report updated benchmarks from enrollment Wave 2 of the study, with an emphasis on curricular areas that align with accreditation standards. METHODS: Seventy doctor of physical therapy (DPT) programs (26.5% national sample) administered the survey to graduates during 2020-2021. Where possible, respondent data were contextualized by statistical comparison with published medical student data (Welch t test, Hedges g). RESULTS: There were 1894 respondents who participated in the study (response rate: 63.9%). The average survey duration was 32.9 minutes. White-only, non-Hispanic/Latino/a/x individuals (78.8%) exceeded the 2020 US Census prevalence (60.1%), and only one-half of respondents perceived a benefit to their training from the diversity present in their programs. Over 94% of respondents indicated that their curricula were characterized by "problem solving/critical thinking" and "clinical reasoning," but nearly one-half indicated "busywork" was prevalent. High curricular satisfaction ratings clustered in content areas relating to profession-specific technical skills and low ratings clustered in foundational sciences. DPT respondents reported significantly lower tolerance for ambiguity, significantly more exhaustion, and significantly less disengagement than medical students. Respondents endorsed higher levels of "adaptive" perfectionism (striving for high performance) than "maladaptive" perfectionism (concern over negative evaluations). Respondents with loans (27.7%) had debt exceeding $150,000, the benchmark above which the DPT degree loses economic power. CONCLUSION: PT-GQ benchmarks revealed strengths (eg, curricula emphasizing problem solving/critical thinking and clinical reasoning) and challenges (eg, low diversity, problematic student debt) in physical therapist education. IMPACT: Programs can use benchmarking for quality-improvement efforts and as a data source for accreditation self-study reports. The ongoing study will refine national benchmarks and pilot items to address new research questions.

Peer review analyze: A novel benchmark resource for computational analysis of peer reviews.

Peer Review is at the heart of scholarly communications and the cornerstone of scientific publishing. However, academia often criticizes the peer review system as non-transparent, biased, arbitrary, a flawed process at the heart of science, leading to researchers arguing with its reliability and quality. These problems could also be due to the lack of studies with the peer-review texts for various proprietary and confidentiality clauses. Peer review texts could serve as a rich source of Natural Language Processing (NLP) research on understanding the scholarly communication landscape, and thereby build systems towards mitigating those pertinent problems. In this work, we present a first of its kind multi-layered dataset of 1199 open peer review texts manually annotated at the sentence level (∼ 17k sentences) across the four layers, viz. Paper Section Correspondence, Paper Aspect Category, Review Functionality, and Review Significance. Given a text written by the reviewer, we annotate: to which sections (e.g., Methodology, Experiments, etc.), what aspects (e.g., Originality/Novelty, Empirical/Theoretical Soundness, etc.) of the paper does the review text correspond to, what is the role played by the review text (e.g., appreciation, criticism, summary, etc.), and the importance of the review statement (major, minor, general) within the review. We also annotate the sentiment of the reviewer (positive, negative, neutral) for the first two layers to judge the reviewer's perspective on the different sections and aspects of the paper. We further introduce four novel tasks with this dataset, which could serve as an indicator of the exhaustiveness of a peer review and can be a step towards the automatic judgment of review quality. We also present baseline experiments and results for the different tasks for further investigations. We believe our dataset would provide a benchmark experimental testbed for automated systems to leverage on current NLP state-of-the-art techniques to address different issues with peer review quality, thereby ushering increased transparency and trust on the holy grail of scientific research validation. Our dataset and associated codes are available at https://www.iitp.ac.in/~ai-nlp-ml/resources.html#Peer-Review-Analyze.

Implant brand portfolios, the potential for camouflage of data, and the role of the Orthopaedic Data Evaluation Panel in total knee arthroplasty.

AIMS: Knee arthroplasty surgery is a highly effective treatment for arthritis and disorders of the knee. There are a wide variety of implant brands and types of knee arthroplasty available to surgeons. As a result of a number of highly publicized failures, arthroplasty surgery is highly regulated in the UK and many other countries through national registries, introduced to monitor implant performance, surgeons, and hospitals. With time, the options available within many brand portfolios have grown, with alternative tibial or femoral components, tibial insert materials, or shapes and patella resurfacings. In this study we have investigated the effect of the expansion of implant brand portfolios and where there may be a lack of transparency around a brand name. We also aimed to establish the potential numbers of compatible implant construct combinations. METHODS: Hypothetical implant brand portfolios were proposed, and the number of compatible implant construct combinations was calculated. RESULTS: A simple knee portfolio with cemented cruciate-retaining (CR) and posterior-stabilized (PS) components, with and without a patella, has four combinations. If there are two options available for each component, the numbers double for each option, resulting in 32 combinations. The effect of adding a third option multiplies the number by 1.3. Introducing compatible uncemented options, with the effect of hybrids, multiplies the number by 4. An implant portfolio with two femoral components (both in CR and PS), with two insert options and a patella, all in cemented and uncemented versions leads to 192 possible compatible implant construct combinations. There are implant brands available to surgeons with many more than two options. CONCLUSION: This study demonstrates that the addition of multiple variants within a knee brand portfolio leads to a large number (many hundreds) of compatible implant construct combinations. Revision rates of implant combinations are not currently reviewed at this level of granularity, leading to the risk of camouflage of true outcomes. Cite this article: Bone Joint J 2021;103-B(10):1555-1560.

Perihilar Cholangiocarcinoma - Novel Benchmark Values for Surgical and Oncological Outcomes From 24 Expert Centers.

OBJECTIVE: The aim of this study was to define robust benchmark values for the surgical treatment of perihilar cholangiocarcinomas (PHC) to enable unbiased comparisons. BACKGROUND: Despite ongoing efforts, postoperative mortality and morbidity remains high after complex liver surgery for PHC. Benchmark data of best achievable results in surgical PHC treatment are however still lacking. METHODS: This study analyzed consecutive patients undergoing major liver surgery for PHC in 24 high-volume centers in 3 continents over the recent 5-year period (2014-2018) with a minimum follow-up of 1 year in each patient. Benchmark patients were those operated at high-volume centers (≥50 cases during the study period) without the need for vascular reconstruction due to tumor invasion, or the presence of significant co-morbidities such as severe obesity (body mass index ≥35), diabetes, or cardiovascular diseases. Benchmark cutoff values were derived from the 75th or 25th percentile of the median values of all benchmark centers. RESULTS: Seven hundred eight (39%) of a total of 1829 consecutive patients qualified as benchmark cases. Benchmark cut-offs included: R0 resection ≥57%, postoperative liver failure (International Study Group of Liver Surgery): ≤35%; in-hospital and 3-month mortality rates ≤8% and ≤13%, respectively; 3-month grade 3 complications and the CCI: ≤70% and ≤30.5, respectively; bile leak-rate: ≤47% and 5-year overall survival of ≥39.7%. Centers operating mostly on complex cases disclosed better outcome including lower post-operative liver failure rates (4% vs 13%; P = 0.002). Centers from Asia disclosed better outcomes. CONCLUSION: Surgery for PHC remains associated with high morbidity and mortality with now the availability of benchmark values covering 21 outcome parameters, which may serve as key references for comparison in any future analyses of individuals, group of patients or centers.

Defining Global Benchmarks in Elective Secondary Bariatric Surgery Comprising Conversional, Revisional, and Reversal Procedures.

OBJECTIVE: To define "best possible" outcomes for secondary bariatric surgery (BS). BACKGROUND: Management of poor response and of long-term complications after BS is complex and under-investigated. Indications and types of reoperations vary widely and postoperative complication rates are higher compared to primary BS. METHODS: Out of 44,884 BS performed in 18 high-volume centers from 4 continents between 06/2013-05/2019, 5,349 (12%) secondary BS cases were identified. Twenty-one outcome benchmarks were established in low-risk patients, defined as the 75th percentile of the median outcome values of centers. Benchmark cases had no previous laparotomy, diabetes, sleep apnea, cardiopathy, renal insufficiency, inflammatory bowel disease, immunosuppression, thromboembolic events, BMI> 50 kg/m2 or age> 65 years. RESULTS: The benchmark cohort included 3143 cases, mainly females (85%), aged 43.8 ±â€Š10 years, 8.4 ±â€Š5.3 years after primary BS, with a BMI 35.2 ±â€Š7 kg/m2. Main indications were insufficient weight loss (43%) and gastro-esophageal reflux disease/dysphagia (25%). 90-days postoperatively, 14.6% of benchmark patients presented ≥1 complication, mortality was 0.06% (n = 2). Significantly higher morbidity was observed in non-benchmark cases (OR 1.37) and after conversional/reversal or revisional procedures with gastrointestinal suture/stapling (OR 1.84). Benchmark cutoffs for conversional BS were ≤4.5% re-intervention, ≤8.3% re-operation 90-days postoperatively. At 2-years (IQR 1-3) 15.6% of benchmark patients required a reoperation. CONCLUSION: Secondary BS is safe, although postoperative morbidity exceeds the established benchmarks for primary BS. The excess morbidity is due to an increased risk of gastrointestinal leakage and higher need for intensive care. The considerable rate of tertiary BS warrants expertise and future research to optimize the management of non-success after BS.

A benchmark for neural network robustness in skin cancer classification.

BACKGROUND: One prominent application for deep learning-based classifiers is skin cancer classification on dermoscopic images. However, classifier evaluation is often limited to holdout data which can mask common shortcomings such as susceptibility to confounding factors. To increase clinical applicability, it is necessary to thoroughly evaluate such classifiers on out-of-distribution (OOD) data. OBJECTIVE: The objective of the study was to establish a dermoscopic skin cancer benchmark in which classifier robustness to OOD data can be measured. METHODS: Using a proprietary dermoscopic image database and a set of image transformations, we create an OOD robustness benchmark and evaluate the robustness of four different convolutional neural network (CNN) architectures on it. RESULTS: The benchmark contains three data sets-Skin Archive Munich (SAM), SAM-corrupted (SAM-C) and SAM-perturbed (SAM-P)-and is publicly available for download. To maintain the benchmark's OOD status, ground truth labels are not provided and test results should be sent to us for assessment. The SAM data set contains 319 unmodified and biopsy-verified dermoscopic melanoma (n = 194) and nevus (n = 125) images. SAM-C and SAM-P contain images from SAM which were artificially modified to test a classifier against low-quality inputs and to measure its prediction stability over small image changes, respectively. All four CNNs showed susceptibility to corruptions and perturbations. CONCLUSIONS: This benchmark provides three data sets which allow for OOD testing of binary skin cancer classifiers. Our classifier performance confirms the shortcomings of CNNs and provides a frame of reference. Altogether, this benchmark should facilitate a more thorough evaluation process and thereby enable the development of more robust skin cancer classifiers.

MuSyC is a consensus framework that unifies multi-drug synergy metrics for combinatorial drug discovery.

Drug combination discovery depends on reliable synergy metrics but no consensus exists on the correct synergy criterion to characterize combined interactions. The fragmented state of the field confounds analysis, impedes reproducibility, and delays clinical translation of potential combination treatments. Here we present a mass-action based formalism to quantify synergy. With this formalism, we clarify the relationship between the dominant drug synergy principles, and present a mapping of commonly used frameworks onto a unified synergy landscape. From this, we show how biases emerge due to intrinsic assumptions which hinder their broad applicability and impact the interpretation of synergy in discovery efforts. Specifically, we describe how traditional metrics mask consequential synergistic interactions, and contain biases dependent on the Hill-slope and maximal effect of single-drugs. We show how these biases systematically impact synergy classification in large combination screens, potentially misleading discovery efforts. Thus the proposed formalism can provide a consistent, unbiased interpretation of drug synergy, and accelerate the translatability of synergy studies.

Proficiency testing for bacterial whole genome sequencing in assuring the quality of microbiology diagnostics in clinical and public health laboratories.

The adoption of whole genome sequencing (WGS) data over the past decade for pathogen surveillance, and decision-making for infectious diseases has rapidly transformed the landscape of clinical microbiology and public health. However, for successful transition to routine use of these techniques, it is crucial to ensure the WGS data generated meet defined quality standards for pathogen identification, typing, antimicrobial resistance detection and surveillance. Further, the ongoing development of these standards will ensure that the bioinformatic processes are capable of accurately identifying and characterising organisms of interest, and thereby facilitate the integration of WGS into routine clinical and public health laboratory setting. A pilot proficiency testing (PT) program for WGS of infectious agents was developed to facilitate widely applicable standardisation and benchmarking standards for WGS across a range of laboratories. The PT participating laboratories were required to generate WGS data from two bacterial isolates, and submit the raw data for independent bioinformatics analysis, as well as analyse the data with their own processes and answer relevant questions about the data. Overall, laboratories used a diverse range of bioinformatics tools and could generate and analyse high-quality data, either meeting or exceeding the minimum requirements. This pilot has provided valuable insight into the current state of genomics in clinical microbiology and public health laboratories across Australia. It will provide a baseline guide for the standardisation of WGS and enable the development of a PT program that allows an ongoing performance benchmark for accreditation of WGS-based test processes.

Disparities in Emergency Medical Services Time Intervals for Patients with Suspected Acute Coronary Syndrome: Findings from the North Carolina Prehospital Medical Information System.

Background Timely emergency medical services (EMS) response, management, and transport of patients with suspected acute coronary syndrome (ACS) significantly reduce delays to emergency treatment and improve outcomes. We evaluated EMS response, scene, and transport times and adherence to proposed time benchmarks for patients with suspected ACS in North Carolina from 2011 to 2017. Methods and Results We conducted a population-based, retrospective study with the North Carolina Prehospital Medical Information System, a statewide electronic database of all EMS patient care reports. We analyzed 2011 to 2017 data on patient demographics, incident characteristics, EMS care, and county population density for EMS-suspected patients with ACS, defined as a complaint of chest pain or suspected cardiac event and documentation of myocardial ischemia on prehospital ECG or prehospital activation of the cardiac care team. Descriptive statistics for each EMS time interval were computed. Multivariable logistic regression was used to quantify relationships between meeting response and scene time benchmarks (11 and 15 minutes, respectively) and prespecified covariates. Among 4667 patients meeting eligibility criteria, median response time (8 minutes) was shorter than median scene (16 minutes) and transport (17 minutes) time. While scene times were comparable by population density, patients in rural (versus urban) counties experienced longer response and transport times. Overall, 62% of EMS encounters met the 11-minute response time benchmark and 49% met the 15-minute scene time benchmark. In adjusted regression analyses, EMS encounters of older and female patients and obtaining a 12-lead ECG and venous access were independently associated with lower adherence to the scene time benchmark. Conclusions Our statewide study identified urban-rural differences in response and transport times for suspected ACS as well as patient demographic and EMS care characteristics related to lower adherence to scene time benchmark. Strategies to reduce EMS scene times among patients with ACS need to be developed and evaluated.

Machine-learning scoring functions trained on complexes dissimilar to the test set already outperform classical counterparts on a blind benchmark.

The superior performance of machine-learning scoring functions for docking has caused a series of debates on whether it is due to learning knowledge from training data that are similar in some sense to the test data. With a systematically revised methodology and a blind benchmark realistically mimicking the process of prospective prediction of binding affinity, we have evaluated three broadly used classical scoring functions and five machine-learning counterparts calibrated with both random forest and extreme gradient boosting using both solo and hybrid features, showing for the first time that machine-learning scoring functions trained exclusively on a proportion of as low as 8% complexes dissimilar to the test set already outperform classical scoring functions, a percentage that is far lower than what has been recently reported on all the three CASF benchmarks. The performance of machine-learning scoring functions is underestimated due to the absence of similar samples in some artificially created training sets that discard the full spectrum of complexes to be found in a prospective environment. Given the inevitability of any degree of similarity contained in a large dataset, the criteria for scoring function selection depend on which one can make the best use of all available materials. Software code and data are provided at https://github.com/cusdulab/MLSF for interested readers to rapidly rebuild the scoring functions and reproduce our results, even to make extended analyses on their own benchmarks.

Current Interventional Radiology-Related Benchmarked Clinical Quality Measures Are Less Likely to be "Capped" Than Diagnostic Radiology Clinical Quality Measures.

In the merit-based incentive payment system (MIPS), quality measures are considered topped out if national median performance rates are ≥95%. Quality measures worth 10 points can be capped at 7 points if topped out for ≥2 years. This report compares the availability of diagnostic radiology (DR)-related and interventional radiology (IR)-related measures worth 10 points. A total of 196 MIPS clinical quality measures were reviewed on the Center for Medicare and Medicaid Services MIPS website. There are significantly more IR-related measures worth 10 points than DR measures (2/9 DR measures vs 9/12 IR measures; P = .03), demonstrating that clinical IR services can help mixed IR/DR groups maximize their Center for Medicare and Medicaid Services payment adjustment.

Dataset for the reporting of carcinoma of the esophagus in resection specimens: recommendations from the International Collaboration on Cancer Reporting.

BACKGROUND AND OBJECTIVES: A standardized data set for esophageal carcinoma pathology reporting was developed based on the approach of the International Collaboration on Cancer Reporting (ICCR) for the purpose of improving cancer patient outcomes and international benchmarking in cancer management. MATERIALS AND METHODS: The ICCR convened a multidisciplinary international expert panel to identify the best evidence-based clinical and pathological parameters for inclusion in the data set for esophageal carcinoma. The data set incorporated the current edition of the World Health Organization Classification of Tumours of the Digestive System, and Tumour-Node-Metastasis staging systems. RESULTS: The scope of the data set encompassed resection specimens of the esophagus and esophagogastric junction with tumor epicenter ≤20 mm into the proximal stomach. Core reporting elements included information on neoadjuvant therapy, operative procedure used, tumor focality, tumor site, tumor dimensions, distance of tumor to resection margins, histological tumor type, presence and type of dysplasia, tumor grade, extent of invasion in the esophagus, lymphovascular invasion, response to neoadjuvant therapy, status of resection margin, ancillary studies, lymph node status, distant metastases, and pathological staging. Additional non-core elements considered useful to report included clinical information, specimen dimensions, macroscopic appearance of tumor, and coexistent pathology. CONCLUSIONS: This is the first international peer-reviewed structured reporting data set for surgically resected specimens of the esophagus. The ICCR carcinoma of the esophagus data set is recommended for routine use globally and is a valuable tool to support standardized reporting, to benefit patient care by providing diagnostic and prognostic best-practice parameters.

Collect Once, Use Many Times: Attaining Unified Metrics for Tuberculosis Preventive Treatment for People Living With HIV.

The World Health Organization (WHO) recommends providing tuberculosis preventive treatment (TPT) to all persons living with HIV and to all household contacts of persons with bacteriologically confirmed pulmonary tuberculosis disease. Regrettably, the absence of a harmonized data collection and management approach to TPT indicators has contributed to programmatic challenges at local, national, and global levels. However, in April 2020, the WHO launched the Consolidated HIV Strategic Information Guidelines, with an updated set of priority indicators. These guidelines recommend that Ministries of Health collect, report, and use data on TPT completion in addition to TPT initiation. Both indicators are reflected in the WHO's list of 15 core indicators for program management and are also required by the US President's Emergency Plan for AIDS Relief's Monitoring, Evaluation, and Reporting (MER) guidance. Although not perfectly harmonized, both frameworks now share essential indicator characteristics. Aligned indicators are necessary for robust strategic and operational planning, resource allocation, and data communication. "Collect once, use many times" is a best practice for strategic information management. Building harmonized and sustainable health systems will enable countries to successfully maintain essential HIV, tuberculosis, and other health services while combatting new health threats.

Editor's Choice - Optimal Threshold for the Volume-Outcome Relationship After Open AAA Repair in the Endovascular Era: Analysis of the International Consortium of Vascular Registries.

OBJECTIVE: As open abdominal aortic aneurysm (AAA) repair (OAR) rates decline in the endovascular era, the endorsement of minimum volume thresholds for OAR is increasingly controversial, as this may affect credentialing and training. The purpose of this analysis was to identify an optimal centre volume threshold that is associated with the most significant mortality reduction after OAR, and to determine how this reflects contemporary practice. METHODS: This was an observational study of OARs performed in 11 countries (2010 - 2016) within the International Consortium of Vascular Registry database (n = 178 302). The primary endpoint was post-operative in hospital mortality. Two different methodologies (area under the receiving operating curve optimisation and Markov chain Monte Carlo procedure) were used to determine the optimal centre volume threshold associated with the most significant mortality improvement. RESULTS: In total, 154 912 (86.9%) intact and 23 390 (13.1%) ruptured AAAs were analysed. The majority (63.1%; n = 112 557) underwent endovascular repair (EVAR) (OAR 36.9%; n = 65 745). A significant inverse relationship between increasing centre volume and lower peri-operative mortality after intact and ruptured OAR was evident (p < .001) but not with EVAR. An annual centre volume of between 13 and 16 procedures per year was associated with the most significant mortality reduction after intact OAR (adjusted predicted mortality < 13 procedures/year 4.6% [95% confidence interval 4.0% - 5.2%] vs. ≥ 13 procedures/year 3.1% [95% CI 2.8% - 3.5%]). With the increasing adoption of EVAR, the mean number of OARs per centre (intact + ruptured) decreased significantly (2010 - 2013 = 35.7; 2014 - 2016 = 29.8; p < .001). Only 23% of centres (n = 240/1 065) met the ≥ 13 procedures/year volume threshold, with significant variation between nations (Germany 11%; Denmark 100%). CONCLUSION: An annual centre volume of 13 - 16 OARs per year is the optimal threshold associated with the greatest mortality risk reduction after treatment of intact AAA. However, in the current endovascular era, achieving this threshold requires significant re-organisation of OAR practice delivery in many countries, and would affect provision of non-elective aortic services. Low volume centres continuing to offer OAR should aim to achieve mortality results equivalent to the high volume institution benchmark, using validated data from quality registries to track outcomes.

Proposed Quality Metrics for Lung Cancer Screening Programs: A National Lung Cancer Roundtable Project.

Lung cancer screening with a low radiation dose chest CT scan is the standard of care for screening-eligible individuals. The net benefit of screening may be optimized by delivering high-quality care, capable of maximizing the benefit and minimizing the harms of screening. Valid, feasible, and relevant indicators of the quality of lung cancer screening may help programs to evaluate their current practice and to develop quality improvement plans. The purpose of this project was to develop quality indicators related to the processes and outcomes of screening. Potential quality indicators were explored through surveys of multidisciplinary lung cancer screening experts. Those that achieved predefined measures of consensus for each of the validity, feasibility, and relevance domains are proposed as quality indicators. Each of the proposed indicators is described in detail, with guidance on how to define, measure, and improve program performance within the indicator.